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SUMMARY 


A variety  of  human  factors  support  services  (e.g.,  function  allocation, 
training  system  development,  cockpit  design)  are  required  in  the  development 
of  naval  air  weapon  systems.  For  many,  perhaps  all,  such  services  adequate 
system  Functional  Requirements  (FRs)  are  a necessary  prerequisite.  Current 
practice  for  preparation  of  FRs,  useful  to  human  factors  engineers,  is  in- 
adequate. One  solution  to  the  problem  is  a manual  of  instructions  for  pre- 
paring adequate  FRs.  This  report  presents  the  consensus  of  opinion  of  a group 
of  experienced  system  developers  about  what  a manual  of  instructions  for 
writing  FRs  should  contain. 
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CONCLUSIONS 


The  opinions  o£  experienced  Human  Factors  Engineers  (HFEs)  confirm 
justifications  provided  in  Subtask  A.l  of  Program  Element  No.  62763N  for  prep- 
aration of  an  instruction  manual  for  writing  Functional  Requirements  (FRs) . 
Briefly,  FRs  are  useful  prerequisites  for  most  human  factors  support  services 
in  system  development.  For  function  allocation,  design  of  equipment  used  by 
humans  and  human  role  definition,  FRs  are  mandatory  prerequisites.  To  be  use- 
ful to  HFEs,  system  FRs  must  be  written  to  permit  more  than  one  means  of 
accomplishing  the  required  functions  (’^implementation  free”  FRs)  . Experienced 
HFEs  agree  a document  of  instructions  for  writing  FRs  could  and  should  be  pre- 
pared. The  document  must:  1.)  define  ’’implementation  free”  FRs,  2.)  provide 
numerous,  specific  examples  of  adequate  and  inadequate  FRs,  3.)  present  and 
suggest  solutions  for  commonly-encountered  problems  in  preparation  of  FRs  and 
4.)  present  clear  steps  to  follow  in  preparing  FRs. 

Human  Factors  Engineers  agree  that  writing  FRs  requires  a cooperative 
effort  between  all  disciplines  involved  in  system  development.  Daily, 
active,  work  interaction,  between  HFEs  and  non-HFEs  (traditional  engineering 
personnel),  was  an  agreed-upon  crucial  source  of  information  for  writing  system 
FRs.  However,  opinions  of  HFEs  revealed  a potentially  serious  problem  with  the 
effectiveness  of  the  interaction.  Human  Factors  Engineers  tend  to  misjudge 
opinions  held  by  non-HFEs  about  HFEs  and  human  factors  support  services '-in 
system  design.  Contrary  to  the  HFEs’  predictions,  opinions  of  non-HFEs  show 
an  understanding  of  and  need  for,  involvement  of  HFEs  and  the  utility  of 
human  factors  support  in  system  design.  If  FRs  are  mandatory  for  at  least  some 
necessary  human  factors  support  services,  if  adequate  FRs  depend  on  effective 
interaction  between  HFEs  and  nHFEs,  and  if  opinions  of  HFEs  about  nHFEs 
reported  here  persist,  even  a well  written  instruction  manual  may  not  be  suffi- 
cient to  insure  preparation  of  FRs  useful  to  HFEs.  Before  an  instruction 
manual  can  be  useful,  HFEs  must  carefully  examine  opinions  they  hold  about, 
and  attribute  to,  nHFEs. 
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INTRODUCTION 


This  document  was  prepared  under  Program  Element  No.  62763N,  Task  Area 
No.  WF55-525-000,  Subtask  A.l  - Tasks  and  Functions  to  be  Allocated  to  Humans: 
Develop  Functional  Requirements  Methods . A Program  Management  Summary  (PMS) , 
supporting  the  Subtask,  provides  supplementary  details  for  the  task  objective. 
Briefly,  the  objective  of  the  task  is  to:  "Prepare  a manual  or  handbook  for 
use  by  Human  Factors  or  non-Human  Factors  engineers  which:  1)  communicates 
the  need  for  the  development  and  use  of  FRs  (Functional  Requirements)  in  the 
design  process;  and  2)  describes  the  procedures  and  methods  to  be  used  for 
the  generation  of  FRs."  Additionally,  the  PMS  states; 

1. )  all  system  design  efforts  require  adequate  "implementation  free" 

FRs, 

2. )  inadequate  FRs  decrease  effectiveness  of  human  factors  engineering 

and  may  lead  to  expensive  and  unnecessary  system  design, 

3. )  criteria  for  distinguishing  between  good  and  bad  FRs  are  needed 

and 

4. )  advice  and  counsel  from  experienced  system  developers  should 

guide  the  effort . 

To  prepare  a preliminary  outline  for  the  proposed  manual,  interviews  with 
senior  members  of  the  Naval  Air  Development  Center  (NADC)  Human  Factors  Division 
(HFD)  were  conducted.  The  interviews  revealed  often  conflicting  and  sometimes 
vague  guidance.  Opinions  from  a larger,  more  diverse  group  and  an  orderly  pro- 
cedure for  obtaining  and  systematizing  the  opinion  were  needed.  The  Delphi 
method  (Quade  § Boucher,  1968)  is  tailor  made  to  fill  the  need. 

The  Delphi  method,  first  developed  for  the  military  by  the  RAND  Corp. 

(Dodge  § Clark,  1977)  is  described  in  detail  elsewhere  (Tinstone  § Turroff,  1975). 
Briefly,  the  Delphi  method  employs  a series  of  questionnaires.  The  first  question- 
naire, usually  requiring  essay  responses  to  a few  general  questions,  is  intended 
to  gather  as  much  information  as  possible  about  a topic.  The  monitor  summarizes 
the  first  returns  and  re-distributes  the  results  as  a second  questionnaire. 

Usually  the  second  and  subsequent  questionnaires  are  in  multiple-choice  format. 

The  third  and  subsequent  questionnaires  include  feedback  to  respondents  by  show- 
ing each  respondent  his  responses  in  comparison  to  the  group's  responses.  Res- 
pondents are  permitted  to  change  their  opinions  and  are  encouraged  to  explain 
any  atypical  opinions.  The  explanations  may  also  be  included  as  feedback  for 
consideration  by  all  respondents.  The  process  continues  until  the  monitor  con- 
cludes he  has  as  much  consensus  of  opinion  as  he  can  get.  The  process  is  con- 
ducted with  assurance  of  anonymity  of  opinion.  Consequently,  each  respondent 
considers  all  opinions  free  of  any  effect  from  association  between  individuals 
and  opinions. 
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"Delphi  may  be  characterized  as  a method  of  structuring  a group  communi- 
cation process  so  that  the  process  is  effective  in  allowing  a group  of  indivi- 
duals, as  a whole,  to  deal  with  a complex  problem."  (Linstone  ^ Turroff,  1975; 
pg.  3).  In  this  case,  the  complex  problem  is  to  determine  the  content  of  an 
instruction  manual  for  writing  FRs . The  appropriate  individuals  to  provide 
advice  and  counsel  are  HFEs  experienced  in  providing  support  services  to  naval 
weapon  system  development.  The  Delphi  method  provides  an  orderly  and  systematic 
procedure  for  determining  consensus  of  opinion  from  the  group  about  the  problem. 
The  opinions  obtained  from  experienced  HFEs  and  presented  in  this  report  are 
intended  as  guidance  to  writers  of  an  instruction  manual  for  writing  FRs  that 
will  be  useful  to  HFEs. 
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METHOD 


Respondents . Sixteen  professional  members  of  the  NADC  Human  Factors 
Division  (HFD)  and  33  members  of  the  NADC  Systems  Department  (SD)  answered 
questionnaires.  HFD  respondents  included  three  Navy  Medical  Service  Corps 
Officers  and  13  civilians.  Eleven  civilian  HFD  respondents  were  Engineering 
Psychologists,  one  was  an  Electrical  Engineer  and  one  a General  Engineer.  SD 
respondents  included  23  Electrical  Engineers,  eight  General  Engineers,  one 
Aerospace  Engineer  and  one  Mechanical  Engineer. 

SD  respondents  claimed  some  work  interaction  with  human  factors  engineers. 
All  respondents  had  at  least  one  year's  experience  in  system  development  work. 
Lastly,  all  respondents  volunteered  their  involvement  and  their  responses 
remained  anonymous  throughout  the  effort . 

Procedure.  A sequence  of  three  questionnaires  was  provided  to  the  HFD 
respondents.  Questionnaire  One  asked  10  general  questions  about  FRs  and  was 
intended  to  stimulate  wide-ranging  essay  responses.  A copy  of  Questionnaire  One 
is  provided  in  Appendix  A. 

Responses  to  Questionnaire  One  were  summarized,  re-written  as  210  single 
sentences  clustered  into  11  general  issues  and  resubmitted  to  HFD  respondents 
as  Questionnaire  Two.  The  11  general  issues  were  presented  in  11  different 
tables,  one  issue  to  a table,  in  Questionnaire  Two.  Each  table  included  a head- 
ing and  a number  of  associated  items.  Respondents  were  instructed  to  combine 
the  heading,  successively,  with  each  item  in  the  table  to  form  sentences.  For 
each  resulting  sentence  respondents  were  asked  to  select  one  of  five  versions 
of  agreement,  viz.,  1)  agree  completely,  2)  tend  to  agree,  3)  uncertain  if 
agree  or  disagree,  4)  tend  to  disagree  and  5)  disagree  completely.  For  example, 
the  heading  for  Table  1 in  Questionnaire  Two  was:  "Functional  Requirements  are:". 
One  item  in  Table  1 was:  "exclusively  a human  factors  construct."  Respondents 
then  formed  the  sentence  "Functional  Requirements  are  exclusively  a human  factors 
construct"  and  selected  one  of  the  five  versions  of  agreement.  Or,  respondents 

could  ignore  any  item  by  not  indicating  a response.  A copy  of  Questionnaire  Two 
is  shown  in  Appendix  B. 

Questionnaire  Three  was  identical  to  Questionnaire  Two  except  response  dis- 
tributions, obtained  from  results  in  Questionnaire  Two,  were  shown  for  each  item 
along  with  the  respondent's  responses  in  Questionnaire  Two.  Thus,  for  each  item, 
respondents  could  compare  their  responses  with  the  responses  of  the  entire  group. 
Questionnaire  Three  provided  respondents  an  opportunity  to  change  responses. 

One  of  the  11  issues  in  Questionnaire  Two  concerned  estimates,  by  human 
factors  engineers  (HFEs),  of  beliefs  held  by  non-human  factors  engineers  (nHFEs) 
about  HFEs  and  human  factors  support  services  in  system  development  (See  Table 
BIO,  Appendix  B) . To  determine  opinions  actually  held  by  nHFEs,  33  nHFEs  from 
the  SD  provided  their  opinions  on  a single  questionnaire  (hence  not  a Delphi 
procedure).  The  questionnaire  used  with  SD  respondents  was  a duplicate  of  one 
table  of  Questionnaire  Two  used  with  the  HFEs  (See  Table  BIO,  Appendix  B)  with 
an  appropriate  change  to  the  table  heading.  Thus,  for  the  issue  concerned  with 
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beliefs  held  by  nHFEs , two  groups  responded  to  the  items.  One  group,  the  HFEs, 
speculated  about  beliefs  held  by  nHFEs.  The  other  group,  the  nHFEs,  stated 
their  beliefs  for  comparison  to  speculations  provided  by  the  HFEs. 
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RESULTS 


Response  distributions  in  Questionnaire  Tliree  for  HFEs  and  response  dis- 
tributions in  the  single  questionnaire  for  nHFEs  represent  group  opinion  for 
each  item.  Group  opinion  represented;  1.)  consensus  of  opinion  for  agreement, 
2.)  consensus  of  opinion  for  disagreement  or  3.)  no  consensus  of  opinion  or 
uncertainty.  Response  distributions  representing  consensus  of  opinion  differed 
in  degree  of  agreement  from  complete  agreement  to  complete  disagreement.  To 
select  distributions  representing  consensus  of  opinion,  observed  distributions 
were  compared  to  distributions  expected  by  chance.  Any  observed  distribution 
with  a probability  of  occurrence,  by  chance,  of  0.140  or  less  was  accepted  as 
representing  consensus  of  opinion  for  the  corresponding  questionnaire  item. 

For  each  distribution  accepted  as  representing  consensus  of  opinion  the  median 
was  computed.  In  data  reported  here  the  median  value  will  fall  on  a scale 
between  1.00  and  5.00.  A median  of  1.00  represents  complete  agreement  while  a 
median  of  5.00  represents  complete  disagreement.  A detailed  discussion  of  the 
analysis  procedure  is  provided  in  Appendix  C.  Response  distributions  for  every 
item  in  the  questionnaires  are  provided  in  Appendix  B,  Results  of  the  analyses 
are  presented  in  Tables  1 through  4. 

Human  factors  engineers  were  allowed  to  change  responses  given  in  Question- 
naire Two  on  Questionnaire  Three.  For  any  response  change,  three  alternative 
outcomes  are  possible:  a.)  change  between  categories  with  equal  frequencies, 

b. )  change  from  a category  with  a lower  to  a category  with  a higher  frequency  or 

c. )  change  from  a category  with  a higher  to  a category  with  a lower  frequency. 
Tlie  case  wherein  a respondent  did  not  respond  to  an  item  may  be  treated  as  if  it 
were  a sixth  category.  Consider  the  following  example  of  a Questionnaire  Two 
distribution : 


6 3 3 2 1 1 


represents  the  category--!  AGREE  COMPLETELY,  represents  — I TEND  TO  AGREE 

and  so  on.  C^  represents  the  case  where  a respondent  did  not  respond  to  the 

item.  Assume  a respondent  had  selected  a response  to  C^  in  Questionnaire  Two. 

If  the  respondent  elects  to  change  his  response  on  Questionnaire  Three,  the 
change  must  be  to  C^,  C^,  C^,  C^  or  C^.  If  he  changes  his  response  to  C^,  the 

change  would  be  classified  as  a change  "From  Lower  to  Higher"  (f=3  to  f=6) . 

If  he  changes  to  the  change  would  be  classified  as  a change  "From  Same  to 

Same"  (f=3  to  f=3) . If  he  changes  to  C , C or  C^,  the  change  would  be  clas- 

4 D o 

sified  as  a change  "From  Higher  to  Lower"  (f=3  to  f=2,  f=l  or  f=l) . Table  5 
presents  the  results  of  the  classification  procedure. 

Response  distributions  for  each  item  represent  group  opinion  about  the  item. 
One  statistical  summary  of  the  group's  opinion  is  the  distribution  median  (the 
point  on  a scale  on  each  side  of  which  there  are  50%  of  the  cases  in  the  dis- 
tribution) . In  Table  10  of  Questionnaire  Three,  HFEs  provided  their  estimate 
of  opinions  held  by  nHFEs  about  HFEs  and  their  role  in  system  development.  The 
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identical  items  of  Table  10  (Questionnaire  Three)  were  provided  to  NHFEs  to 
obtain  their  opinions  about  HFEs  and  the  role  of  HFEs  in  system  development. 

Table  6 shows  items  for  which  a statistically  significant  (two-tailed)  difference 
between  medians  was  observed  by  application  of  the  Medians  Test  (Siegel,  1956) 
to  each  pair  of  distributions.  In  Table  6 items  are  presented  in  order  from 
the  largest  to  the  smallest  difference  between  distribution  medians. 
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Table  1 

Human  Factors  Engineers  Agree 


FRs  are: 

Item  Median 

useful  if  they  facilitate  the  work  for  which  they  are  needed*  1.17 

mandatory  in  system  development  since  they  and  they  alone  are 
fair  standards  by  which  the  adequacy  of  the  system  can  be 

evaluated.  1.23 

mandatory  for  development  of  any  system  when  at  least  one  human 

is  to  be  a component  of  the  system.  1.25 

useless  if  they  do  not  facilitate  the  work  for  which  they  are 

needed.  1.23 

statements  of  WRAT  a system  must  accomplish.  1.23 

"activities  a system  must  perform  independent  of  the  means  of 
accomplishing  the  activities."  L.30 

"what  a thing  (system,  display,  control)  is  to  accomplish."  1.30 

"mandatory  attributes  that  a system  must  possess  to  accomplish 

its  defined  mission."  1,39 

written  to  a level  of  detail  determined  by  the  use  to  which  the 
functional  requirements  will  be  put.  1,50 

presented  in  hierarchical  form  starting  with  general  and  working 
to  specific  requirements.  1.75 

seldom,  if  ever,  adequately  prepared.  1.75 

the  end  product  of  a procedure  or  process.  1.81 

always  a means  to  an  end.  1.83 

never  written  once  in  the  development  of  a system;  rather  they 
are  written  at  the  start  and  continuously  refined  as  more  infor- 
mation becomes  available.  1.95 

related  to  the  concept  of  "degrees  of  freedom"  in  that  as  equipment 
is  ever  more  precisely  defined,  the  corresponding  loss  in  alternative 
approaches  to  accomplish  system  functions  is  similar  to  a reduction 
in  "degrees  of  freedom."  1,95 
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Table  1 (Continued) 


FRs  should  be  written  bv: 

Item 

Median 

a group  of  writers  that  represent  all  the  different  interests 
in  the  system  development  process. 

1.93 

someone  who  understands  thoroughly  the  warfare  area  (e.g.,  ASW, 
air-to-air  combat)  for  which  the  system  is  being  developed. 

1.96 

Likely  sources  of  information  for  writing  FRs  include; 

Operational  Requirements . 

1.39 

Specific  Operational  Requirements. 

1.50 

interviews  with  non-human  factors  engineers  directly  involved 
in  the  system  development  process. 

1.92 

statements  of  Navy  needs. 

1.94 

experience  gained  by  direct,  daily  involvement  of  the  writer (s) 
with  non-HF  engineers  on  the  development  team. 

2.05 

Adequate  FRs  are  needed  for; 

function  allocation  procedures. 

1 

.08 

panel  design. 

1 

. 13 

control  design. 

1 

.13 

display  design. 

1, 

.13 

operator  role  definition. 

1, 

,13 

crew  station  design. 

1, 

.18 

maintainer  role  definition. 

1. 

.25 

crew  size  determinations. 

1. 

,25 

functional  flow  block  diagramming. 

1. 

33 

task  analyses. 

1. 

33 

system  test  and  evaluation. 

1. 

33 

operational  sequence  diagramming. 

1. 

56 
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Table  1 (Continued) 


Adequate  FRs  are  needed  for: 

Item 

Median 

selection  of 

equipment  components . 

1.56 

construction 

of  time  line  analyses. 

1.75 

FRs  should  be  written  without  reference  to : 

outcomes  of  an  implied  function  allocation  process.  Thus, 

"Accomplish  MK  46  torpedo  launch"  is  a better  FR  than  "Pilot 

accomplish  MK  46  torpedo  launch."  1.83 

specific  human  responses.  Thus,  "Pilot  launch  MK  46  torpedo" 

is  a better  FR  than  "Pilot  press  LAUNCH  button  on  a torpedo 

control  panel."  1.83 


Tlie  probability  adequate  FRs^  can  be  written  depends , in  part,  on: 

point  in  the  development  process  when  FRs  are  written — the  earlier 

in  the  process,  the  higher  the  probability  adequate  FRs  can  be 

written.  1.63 


^ document  of  instructions  for  writing  FRs : 

if  well  written  and  understandable  to  individuals  in  system  development, 
would  be  useful.  1.50 

could  be  written.  1.86 


A document  of  instructions  for  writing  FRs  should  include  ^ section : 

clearly  defining  functional  requirements  and  the  general  issues 

involved  in  writing  good  ones.  1.17 

of  a generous  number  of  examples  of  adequate,  marginal  and  in- 
adequate FRs  along  with  explanations  why  each  of  the  ^examples  was 
classified  as  it  was.  1.30 

describing  the  kinds  of  problems  typically  encountered  in  writing 
functional  requirements  along  with  suggestions  for  overcoming  the 
problems.  1.39 


9 


NADC-78076-60 


Table  1 (Continued) 

A ^cment  of  instructions  for  writing  FRs  should  include  a section: 


Item 


providing  an  example  of  how  an  existing  system  evolved  and  where 
in  the  evolution  functional  requirements  would  have  been  needed, 
what  the  functional  requirements  should  have  looked  like  and  how 
they  vtfould  have  been  used  by  human  factors  engineers . 


Median 


1.50 


describing  the  various  uses  HF  engineers  have  for  functional  require 
ments . 


on  the  need  for  clarity  in  thinking  and  writing  along  with 
suggestions  and  references  for  achieving  clarity. 


1.90 

1.94 


nHFEs  believe  HFEs : 

should  not  have  final  authority  in  any  decision  in  the  development 
process. 

provide  services  that  cost  too  much  money  to  be  worthwhile  in  the 
development  process. 

do  not  appreciate  the  speed  with  which  things  happen  during  the 
system  development  process. 

can't  respond  to  system  development  problems  without  first  pro- 
posing a lengthy  "investigation." 


should  be  involved  in  the  system  development  process  but  only  after 
major  equipment  components  have  been  selected. 


1.83 


1.88 

1.95 

2.00 

2.00 


HFEs  believe  nHFEs : 

are  biased  to  think  in  terms  of  equipment  solutions  to  system 
functions . 

do  not  appreciate  the  complexity,  time  and  effort  involved  in 
human  factors  support  services . 

are  quick  to  cut  funds  for  human  factors  support  services  from 
the  budget  when  funding  is  cut . 

assume  humans  will  adapt  to  the  demands  of  the  man-machine  interface 
since  humans  have  always  done  so  in  the  past. 

assume  training  will  take  care  of  helping  humans  adapt  to  any 
demands  the  system  might  impose. 
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Table  1 (Continued) 


HFEs  believe  nHFEs : 

Item 

Median 

will  abandon  even  an  excellent  HF  engineering  suggestion  if 
the  suggestion  interferes  with  a constraint  imposed  by  the  needs 
of  the  equipment . 

1.88 

are  unconvinced  about  the  usefulness  of  an  HF  discipline. 

1.93 

are  willing  to  give  only  ’’lip  service^*  to  the  need  for  HF 
involvement  in  the  system  development  process. 

1.93 

assign  as  many  functions  as  possible  to  equipment  and  assign 
to  humans  only  what  unavoidably  remains. 

1.94 

do  not  know  how  and  why  HF  engineers  do  what  they  do . 

2.00 

do  not  feel  any  responsibility  for  the  adequacy  of  human  performance 
in  the  developed  system  when  it  is  delivered  to  the  fleet. 

2.00 

are  afraid  HF  engineers  will  interfere  with  the  non-HF  engineer's 
prerogative  to  make  system  design  decisions. 

2.08 

are  convinced  the  HF  discipline  has  some,  but  a vague,  role  in  the 
system  development  process. 

2.12 

want  to  develop  new  systems  by  improving  the  old  ones. 

2.12 
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Table  2 

Human  Factors  Engineers  Disagree 


FRs  are : 

Item  Median 

statements  of  IVHEN  a system  must  accomplish  something.  4.20 

useful  only  in  some  subsequent  human  factors  support  service.  4.61 

exclusively  a human  factors  construct.  4.88 

FRs  should  be  written  by : 

almost  anyone  who  wants  to  write  functional  requirements  since 

the  process  is  relatively  simple  once  you  know  what  is  required 

and  considered  adequate.  4.25 

Likely  sources  of  information  for  writing  FRs  include : 
outcomes  of  a function  allocation  procedure.  4.14 

FRs  should  be  written  without  reference  to : 

equipment.  Thus  "destroy  enemy  forces"  is  a better  FR  than  "Destroy 
enemy  submarines."  4.50 

A document  of  instructions  for  writing  FRs : 

is  not  worth  the  effort  required  to  produce  it.  4.06 

is  impossible  to  write  since  the  procedure  varies  too  much  from 

one  system  to  another.  4.13 

is  not  needed  since  the  procedure  for  writing  functional  require- 
ments is  an  art  similar  to  writing  musical  masterpieces  and  thus 
cannot  be  described.  4.17 

is  not  needed  since  the  procedure  for  writing  functional  require- 
ments is  well  known.  4.25 
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Table  2 (Continued) 


HFEs  believe  nHFEs: 

Item 


Median 


should  be  reduced  to  a very  low  level  in  the  decision-making 
process  in  system  development. 


4.00 
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Table  3 


Non-Human  Factors  Engineers  Agree 


HFEs: 

Item 

Median 

have  a useful  role  in  the  system  development  process. 

1.42 
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Table  4 

Non-Human  Factors  Engineers  Disagree 


HFEs: 

Median 

try  to  "horn  in"  on  a process  where  they  are  not  wanted.  4.02 

don't  provide  anything  more  than  common-sense  suggestions 

to  the  system  development  process.  4.13 

provide  services  that  cost  too  much  money  to  be  worthwhile 

in  the  development  process.  4.25 

should  be  involved  in  the  system  development  process  but  only 

after  major  equipment  components  have  been  selected.  4.26 

are  arrogant.  4.38 

are  useful  only  in  support  services  (e.g.,  training  system 
design,  test  and  evaluation)  after  prototype  models  of  the 

system  have  been  built.  4.47 

should  be  restricted  to  designing  comfortable  and  attractive 

crew  stations.  4.58 

have  little  or  nothing  of  value  to  contribute  to  the  system 

development  process.  4.75 
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16 

19 

24 

28 

30 

32 

37 

39 

41 

53 

66 

69 

81 

84 

91 
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Table  5 


Response  Changes 


Change  From 


Lower  to  Higher  to  Same  to 

Higher  Lower  Same 

Frequency  Frequency  Frequency 


0 

21 

26 

6 

15 

10 

5 

9 

15 

4 

12 

20 

4 

7 

7 

3 


0 

2 

1 

0 

3 

5 
0 
7 
2 
0 

6 
0 
0 
1 
1 
2 


0 

2 

2 

0 

2 

0 

2 

0 

1 

0 

3 

0 

0 

0 

0 

0 
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Table  6 


I 


Differences  of  Opinion  Between  Human  Factors  Engineers  (IIFEs)  and  non-IIuman  Factors  Engineers  (nllFEs) 

(Medians  of  nllFEs ' data  predicted  by  HFEs  (arrow  down)  vs.  Medians  of  data  provided  by  nllFEs  (arrow  up)) 

~~~  Agree  Disagree 

Items  0 • 5 ^ 3. 0 5 . 5 

Human  Factors  Engineers : 

**  HFEs 

provide  services  tliat  cost  too  much  money  to  be  worthwhile  in  + +-'3C-__  + + + 

the  development  process,  nHFEs 

should  be  involved  in  the  system  development  process  but  only  after *  **  +■ + + + 

major  equipment  components  have  been  selected. 

are  useful  only  in  support  services  (e.g.,  training  system  design,  test  **  +'•-• ^ + 

and  evaluation)  after  prototype  models  of  the  system  have  been  built. 

provide  services  that  are  too  time  consuming  to  be  worthwhile  in  the  **  +-■ +__'sr-_+ + + 

development  process. 

have  a useful  role  in  the  system  development  process.  **  + ^ + + + 

don't  provide  anything  more  than  common-sense  suggestions  to  the  system  **  + „+__'sk'_+ + __^^  + + 

development  process. 

should  be  restricted  to  designing  comfortable  and  attractive  crew  **  + + + + 

stations . 

are  useful  only  to  suggest  arrangements  for  the  location  of  controls  **  + + — + --^--  + + 

and  displays  at  the  man-machine  interface. 

have  little  or  nothing  of  value  to  contribute  to  the  system  development  **  + + + _'3»r__+ + 

process . 

overemphasize  the  role  of  the  human  in  the  system.  **  + + _-Z^+ + _^__  + + 

may  be  O.K.,  but  the  ones  engineers  have  actually  worked  with  did  **  + + + + 

not  live  up  to  expectations. 
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Table  6 (Continued) 


Items 


Agree  Disagree 

0.5  3.0  5.5 


Human  Factors  Engineers : 

try  to  "horn  in"  on  a process  where  they  are  not  wanted, 
promise  a great  deal  but  deliver  very  little. 

should  not  have  responsibility  for  deciding  if  a human  or  a machine 
should  accomplish  a given  system  function. 

are  unwilling  to  "work  in  the  trenches"  with  working  level  non-human 
factors  engineers . 

treat  each  system  problem  they  are  asked  to  help  resolve  as  if  it 
were  the  first  time  the  problem  had  ever  come  up. 

are  welcomed  in  the  system  development  process. 

use  a lot  of  jargon  that  even  other  human  factors  engineers  do  not 
really  understand. 

are  responsible  for  any  inadequacies  in  communication  or  cooperation 
between  non-human  factors  and  human  factors  engineers . 

should  not  have  final  authority  in  any  decision  in  the  development 
process . 

are  unaware  of  the  factors  that  really  determine  events  and  decisions 
in  the  system  development  process. 

are  insensitive  to  demands  and  responsibilities  non-human  factors 
engineers  face  in  system  development. 

can't  respond  to  system  development  problems  without  first  proposing 
a lengthy  "investigation." 

**  indicates  p = 0.01 
* indicates  p = 0.05 


+ - + + - - + + 


**  + 


^ + + 


+ + 


**  + + + + 


**  + + + + 


-r--+  - 


+ + + + 
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DISCUSSION 


Discussion  about  the  procedure , The  procedure  used  in  this  exercise  with 
HFEs  was  a variation  of  the  traditional  Delphi  procedure.  The  amount^  type 
and  form  of  feedback  provided  differed  from  that  used  in  most  applications  of 
the  Delphi  procedure  (See  Linstone  § Turroff , 1975  for  examples) . HFEs  were 
provided  feedback  once  between  a second  and  third  (here  also  last)  question- 
naire. HFEs  were  asked,  in  the  third  questionnaire,  to  compare  their  responses 
to  responses  of  the  group.  Respondents  could  change  their  responses  after 
noting  how  the  group  had  responded.  Explanations  of  atypical  opinions  were 
not  requested;  nor,  if  volunteered,  used.  The  procedure  here  was  a compromise 
between  a major  advantage  of  the  Delphi  procedure,  viz.,  ultimate  arrival  at 
maximum  consensus  of  opinion  of  the  group,  and  a major  disadvantage,  viz.,  the 
time  and  effort  required  from  respondents. 

In  Questionnaire  Three,  respondents  indicated  response  changes  from  responses 
in  Questionnaire  Two.  Fifteen  of  16  HFE  respondents  made  changes  (See  Table  5). 
Table  5 shows  a majority  (162  of  206)  of  the  changes  were  from  a category  with 
a lower  response  frequency  to  a category  with  a higher  response  frequency.  If 
feedback  provided  in  this  exercise  prompted  changes  of  opinion,  results  in 
Table  5 show  opinion  was  steered  toward  greater  consensus.  For  nHFE  respondents, 
only  a single  questionnaire,  hence  not  a Delphi  procedure,  was  used. 

Discussion  about  what  Functional  Requirements  (FRs)  are.  The  following  dis- 
cussion is  intended  to  structure  results  shown  in  Tables  1 and  2 under  the 
heading  FRs  are. 

Discussion  is  restricted  to  FRs  for  Naval  air  weapon  systems.  A Naval  air 
weapon  system  may  be  defined  as  a collection  of  interdependent  components  serving 
a common  purpose  of  attack  or  defense  in  a potential  or  actual  fight  predominantly 
involving  Navy  people  and  airborne  equipment.  Such  systems  exist  on  a continuum 
extending  from  conception  to  retirement  from  service  use.  For  convenience  in 
subsequent  argument,  the  continuum  may  be  divided  into  two  parts.  The  earlier 
part  can  be  called  the  pre-deployment  phase  and  includes  the  traditional:  a.) 
Program  Initiation,  b.)  Full  Scale  Development,  and  c.)  Production  Phases  common 
in  Weapon  System  Development.  The  later  part  may  be  called  the  deployment  phase. 
The  deployment  phase  begins  with  fleet  introduction  and  ends  with  retirement 
from  service  use. 

For  a system  well  into  the  deployment  phase,  it  would  be  possible  to  con- 
struct a hierarchical  list  of  all  (man  and  machine)  components.  Hierarchical 
listing  begins  with  major  components.  Major  components  are  sub-divided  into 
sub- components , sub-components  into  their  sub -components  and  so  on.  The  result- 
ing list  illustrates  what  can  be  called  structural  interdependence  since  removal 
of  any  component  from  the  list  removes  all  of  that  component's  sub-components. 

For  example,  if  the  component--Pilot--is  removed,  all  the  Pilot *s  sub-components 
(e.g.,  visual  system)  are  also  removed.  The  Pilot  and  his  visual  system  are 
structurally  interdependent. 
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The  proposed  list  could  be  written  to  almost  any  degree  o£  explicitness. 

The  degree  of  explicitness  selected  depends  on  who  will  use  the  list  for  what 
purpose.  Thus,  one  degree  of  explicitness  is  appropriate  if  one  is  to  describe 
the  system  to  attendees  at  a Rotary  Club  luncheon  and  another  degree  of  explicit- 
ness if  one  is  responsible  for  ordering  all  necessary  replacement  components 
for  the  system  during  its  use. 

The  degree  of  explicitness  selected  depends  on  availability  of  information 
about  a component.  For  example,  one  could  list:  Operator  or  Pilot  or  Helo 
Pilot  or  H-2F  Pilot  and  so  on  to  LCDR.  John  Smith.  Clearly,  one  could  not  list 
John  unless  one  knew  John  would  be  a component  of  the  system.  Hence,  availa- 
bility of  information  sets  limits  on  the  degree  of  explicitness  one  may  select. 
Note  however,  because  information  is  available  does  not  mean  it  must  be  used. 

The  choice  between  listing--Pilot--and--LCDR.  John  Smith--  depends,  as  before, 
on  who  is  making  the  list  for  what  purpose.  In  any  case,  the  main  point  is 
the  proposition  that  maximum  degree  of  explicitness  is  possible  for  a system 
well  into  the  deployment  phase  because  maximum  amount  of  information  about  the 
system  is  available. 

For  each  item  in  the  list  it  must  be  possible  to  list  an  associated 
function  or  functions.  Since,  if  for  any  component  no  function  can  be  specified, 
the  component  does  not  rightfully  belong  in  the  list.  Awful  problems  are  en- 
countered in  listing  functions.  Some  components  have  many  functions.  Identical 
components  may  have  different  functions  while  different  components  may  have 
identical  functions.  And,  the  problems  are  especially  acute  in  listing  diverse 
functions  of  human  components.  It  is  not  suggested  here  that  listing  functions 
is  quick,  easy,  cheap  or  fun.  It  is  argued  only  that  it  must  be  possible  to 
list  them.  The  difficulty  of  the  task  is  irrelevant  to  the  argument. 

If  functions  are  associated  with  each  component  in  a hierarchical  list  of 
components,  the  functions  are  also  interdependent.  This  interdependence  may  be 
called  functional  interdependence.  Thus,  for  example,  failure  of  a lock  washer 
under  a machine  screw  in  the  power  amplifier  of  a Radar  Receiver  and  so  on  up 
the  hierarchy,  should  result  in  failure  (ignoring  the  concept  of  degraded  modes 
of  operation)  of  one  or  more  system  functions.  Seldom  is  functional  inter- 
dependence so  tightly  knit  that  failure  of  a component,  at  the  very  bottom  of 
a list  written  to  a very  high  degree  of  explicitness,  inevitably  results  in 
system  failure.  But,  accident  analyses  sometimes  uncover  surprising  examples 
of  tight  functional  interdependence.  In  any  case,  the  function  of  our  example 
lock  washer  and  the  functions  of  the  system  are  interdependent. 

A statement  of  functions  for  any  component  of  a system  is  incomplete  unless 
all  conditions  that  can  influence  successful  accomplishment  of  the  functions 
are  identified.  As  with  functions,  style,  format  and  degree  of  explicitness 
for  itemizing  conditions  are  optional  and  probably  reflect  their  intended  use. 

But,  all  conditions  must  be  identified  in  whatever  scheme  is  used.  Consequences 
of  ignoring  any  condition  are  directly  related  to  frequency  of  occurrence  of 
the  condition  and  criticality  of  the  function  during  system  use.  Incidentally, 
a distinction  between  function  and  condition  depends  on  the  cataloging  scheme  used. 
For  example,  a function  may  be--land  aircraft  on  Carrier  flightdeck.  Alter- 
natively, the  function  may  be — land  aircraft--and  associated  conditions  may 
include--on  a Carrier  flightdeck.  This  illustration  of  variation  in  cataloging 
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schemes  should  not,  however,  obscure  the  important  issue,  viz.,  statements  of 
functions  are  incomplete  unless  all  conditions  affecting  successful  accomplish- 
ments of  the  function  are  identified. 

To  this  point,  argument  has  attempted  to  convince  the  reader  that  a hier- 
archical list  of  all  components  of  a system  well  into  the  deployment  phase  could 
be  written.  For  each  component  listed,  one  or  more  functions  could  be  identified. 
For  each  function  all  conditions  potentially  affecting  accomplishment  of  the 
function  must  accompany  the  function. 

Consider  next  determinants  of  conditions  affecting  functions.  At  least 
two  sources  may  be  identified.  One  source  is  "higher  authority,"  the  other 
is  from  the  components  themselves.  Navy  management  sets  various  conditions 
on  system  functions  chiefly  based  on  time,  money  and  anticipated  threat  con- 
siderations. Since  concern  here  is  with  an  existing  system,  it  is  reasonable 
to  assume  a set  of  necessary  conditions  provided  by  "higher  authority"  must 
have  been  available  at  one  time  in  the  development  of  the  system.  In  this 
argument,  conditions  and  functions  assigned  by  "higher  authority"  are  not  issues 
for  debate,  they  are  given.  Then  too,  conditions  imposed  by  components  are  far 
more  important  to  this  discussion.  Such  conditions  are  imposed  by  capabilities 
of  the  component  itself.  For  example,  for  the  component--Pilot--one  function 
may  be--land  aircraft.  Conditions  associated  with  the  function  may  include: 
a.)  at  night,  b.)  on  a Carrier  flightdeck,  c.)  in  a state  seven  sea  and  so  on. 

Now,  if  one  assumes  the  component--Pilot--can  accomplish  the  function  under 
specified  conditions,  the  component,  function  and  conditions  are  in  accord.  If 
however,  the  component--Pilot--were  instead- -ENS . Jack  Smith--it  is  possible 
Jack  could  land  the  aircraft  on  a Carrier  flightdeck  at  night,  but  only  in  a 
flat  sea.  We  must  either  replace  Jack  or  alter  the  condition — in  a state 
seven  sea.  The  direction  we  go  will  depend  on  whether  we  permit  components  to 
determine  functions  or  functions  to  determine  components.  The  important  point 
for  this  discussion  is  that  components  set  unique  conditions  (hereafter  called 
constraints)  on  functions  and  are  separable  from  conditions  imposed  by  other 
sources . 

When  applied  to  a system  well  into  the  deployment  phase,  the  process 
described  above  could  produce  a hierarchical  list  of  all  components,  associated 
functions  and  for  each  function  an  associated  list  of  conditions  and  associated 
constraints . 

Now,  if  one  discards  the  list  of  components  and  the  associated  constraints, 
what  are  left  are  "implementation  free"  functional  requirements  for  the  system 
under  consideration. 

The  foregoing  operational  definition  of  "implementation  free"  FP.s  reflects 
one  interpretation  of  the  results  shown  in  Tables  1 and  2 under  the  heading  FRs 
are.  To  identify  and  discuss  implications  of  the  definition  is  not  the  purpose 
of  this  report.  Rather,  the  purpose  is  to  provide  one  possible  definition, 
based  on  an  interpretation  of  opinions  of  experienced  HFEs . Writers  of  the 
manual  or  guidebook  may  accept,  modify  or  reject  the  definition  above;  they 
cannot  however,  escape  the  need  for  including  some  precise  definition  of  "im- 
plemenation  free"  FRs  in  the  final  docxment. 
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Discussion  about  who  should  write  FRs.  Consensus  of  opinion  among  HFEs  is 
that  FRs  should  be  written  by  a group  of  writers  that  represent  all  the  different 
interests  in  the  system  development  process  (See  Table  1).  This  finding  is 
opposed  to  a suggestion  in  the  Program  Management  Summary  that  nHFEs  be  res- 
ponsible for  writing  FRs  adequate  for  use  by  HFEs.  Furthermore,  HFEs  agree 
writers  of  FRs  must  understand  thoroughly  the  warfare  area  (e.g.,  ASW)  for  which 
the  system  is  being  developed  (See  Table  1).  Lastly,  HFEs  disagree  that  the 
process  of  writing  FRs  is  relatively  simple  and  can  be  followed  by  the  inexperi- 
enced (See  Table  2) . 

Discussion  about  likely  sources  o^  information  for  writing  FRs.  Table  1 
shows  written  and  verbal  sources  of  information.  The  written  sources  cited 
(e.g..  Operational  Requirements)  are  documents  characteristic  of  the  earliest 
phases  in  system  development.  Verbal  sources  include  interviews  with  nHFEs  and 
the  communication  involved  in  direct,  daily  involvement  with  nHFEs  in  their 
working  environment.  Taken  together,  the  results  suggest  adequate  FRs  can  not 
be  written  by  HFEs  simply  from  documents  routinely  provided  during  system  develop- 
ment; rather,  FRs  should  be  written  by  a team  of  individuals,  representing 
various  skills,  in  active  daily  contact  during,  especially,  the  early  part  of 
the  system  development  process. 

Discussion  about  human  factors  support  s^rvi^es  for  which  adequate  FRs 
are  needed . In  Table  5 of  Questionnaire  Two  (See  Appendix  B) , 19  human  factors 
support  services  are  listed.  Table  1 shows  HFEs  agree  that  adequate  FRs  are 
required  for  14  of  the  19.  Table  1 further  shows  HFEs  agree  completely  (Mdn. = 
1.49)  that  adequate  FRs  are  required  for  11  of  the  14.  The  results  support  the 
notion  that  adequate  FRs  are  a necessary  prerequisite  for  almost  all  human 
factors  support  services  to  weapon  system  development. 

Discussion  about  £ document  of  instructions  for  writing  FRs . Results  in 
Tables  1 and  2 under  the  heading:  A document  of  instructions  for  writing  FRs: 
may  be  summarized  as  follows.  The  document  is:  1.)  needed,  2.)  possible,  3.) 
worth  the  effort  required  to  produce  it  and  4.)  if  well  written,  would  be  useful. 
The  results  provide  support,  from  experienced  HFEs,  for  the  established  Work 
Unit . 


Discussion  about  content  ^ the  document.  Table  1 shows  HFEs  agree  a use- 
ful document  should  include:  1.)  a clear  definition  of  FRs,  2.)  a discussion  of 
the  issues  involved  in  writing  good  FRs  and  3.)  a description  of  the  problems 
commonly  encountered  in  writing  FRs  and  suggested  solutions  for  resolving  the 
problems.  Table  1 further  shows  HFEs  recommend  the  document  provide  sufficient 
specific  examples  to  illustrate  the  differences  between  adequate  and  inadequate 
FRs. 


Discussion  about  opinions  held  by  one  group  about  the  other.  Results  in 
Tables  1,  2,  3,  and  4,  under  the  headings  nHFEs  believe  HFEs,  HFEs  believe  nHFEs 
and  HFEs  represent  consensus  of  opinion  for  predictions  by  or  actual  opinions  of, 
one  group  about  the  other.  Taken  together,  the  results  show:  1.)  HFEs  predict 
nHFEs  hold  HFEs  and  human  factors  support  services  in  low  esteem,  2.)  HFEs  hold 
nHFEs  and  the  roles  of  nHFEs  in  low  esteem  and  3.)  nHFEs  do  not  hold  HFEs  and 
human  factors  support  services  in  low  esteem.  For  items  taken  to  represent  con- 
sensus of  opinion  (Tables  1 through  4),  HFEs'  predictions  of  nHFEs'  opinions  are 
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consistently  inaccurate. 

The  extent  o£  differences  of  opinion  between  HFEs  and  nHFEs  may  be  high- 
lighted by  comparison  of  all  items  considered  by  both  groups  regardless  of 
whether  or  not  consensus  of  opinion  was  observed.  Table  6 presents  23  of  34 
items,  considered  by  both  groups,  for  which  a statistically  significant  (g^-OS) 
difference  between  distribution  medians  was  observed.  The  result  in  Table  6 
provides  support  for  the  conclusion  that  HFEs  generally  predict  that  nHFEs 
hold  HFEs  and  human  factors  support  services  in  low  esteem  and  that  the  HFEs 
are  generally  wrong.  Various  explanations  for  the  differences  in  opinion  are 
possible.  However,  assuming  the  beliefs  expressed  in  Table  6 are  genuine,  a 
single  conclusion  appears  self-evident.  To  produce,  by  cooperative  effort 
among  HFEs  and  nHFEs,  system  FRs  adequate  for  use  by  HFEs,  the  HFEs  must  first 
alter  their  inaccurate  beliefs. 
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QUESTIONNAIRE  ONE 


Please  provide  your  responses  on  sheets  of  paper  and  on  each  sheet  print 

your  name  and  identify  your  response  with  the  appropriate  item  number. 

Item 

1.  Try  to  provide  a useful  definition  of  functional  requirements.  If  you 
cannot,  please  tell  me  why  you  cannot. 

2.  Have  you  provided  any  human  factors  support  to  one  or  more  of  the  follow- 
ing systems:  LAMPS,  VPX,  F-18,  VSTOL,  PROTEUS,  CVTSC,  P3C  Update,  KCX? 

If  yes,  which  system(s)  and  what  kind  of  support  did  you  provide? 

3.  Have  you  encountered  difficulty  with  inadequate  functional  requirements  in 
your  human  factors  work?  If  yes,  please  describe  the  difficulties  and  the 
circums tances . 

4.  Do  you  think  you  could  distinguish  between  adequate  and  inadequate  functional 
requirements?  If  yes,  what  criteria  would  you  use  to  make  the  distinction? 

5.  Construct  any  useful  example  and  describe  the  process  you  would  follow  to 
develop  adequate  functional  requirements.  If  possible,  using  your  example, 
provide  the  functional  requirements  resulting  from  the  process. 

6.  Development  of  functional  requirements  probably  involves  communication  and 
cooperation  among  managers,  non-human  factors  engineers  and  human  factors 
engineers.  Assume  poor  communication  and  lack  of  cooperation  among  the 
mix  of  talent  interferes  with  development  of  adequate  functional  require- 
ments. Please  describe  the  most  likely  kinds  of  communication/cooperation 
problems  and  who  would  be  involved.  Then,  provide  any  suggestions  to  relieve 
the  problems  you  suspect. 

7.  Assume  you  were  asked  to  write  a manual  or  handbook  for  use  by  someone  to 
write  adequate  functional  requirements.  Do  you  think  such  a document  could 
be  written?  If  no,  why  not?  If  yes,  would  anyone  in  the  system  design 
process  be  able  to  use  the  document  to  produce  adequate  functional  require- 
ments? If  no,  who  would  be  ruled  out?  Should  non-human  factors  engineers 
write  the  functional  requirements?  If  no,  who  should?  For  the  last  two 
questions,  please  state  the  reason  for  your  choice.  Lastly,  do  you  know  if 
such  a document  exists?  If  yes,  where  could  I find  it? 

8.  When  in  the  system  development  process  (from  concept  formulation  to  retire- 
ment of  the  system  from  the  fleet)  are  the  functional  requirements  needed? 
What  human  factors  support  service  would  require  them? 

9.  What  information  would  you  need  before  you  could  produce  adequate  functional 
requirements?  If  it  isn’t  obvious,  where  would  you  get  the  information? 
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Item 

10. 


QUESTIONNAIRE  ONE  (Continued) 


Here  please  provide  any  comments  or  suggestions  you  consider  useful  to 
me  to  help  understand  functional  requirements  and  their  problems. 
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Table  B1 

Functional  requirements  are: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

1. 

a list  of  sentences. 

4 

8 

1 

1 

2 

2 . 

one  or  more  expository  paragraphs 

5 

5 

0 

3 

3 

5 . 

presented  in  hierarchical  form  starting  with  general 
and  working  to  specific  requirements . 

6 

8 

2 

0 

0 

4. 

exclusively  a human  factors  construct. 

0 

0 

1 

2 

13 

5 . 

the  end  product  of  a procedure  or  process. 

5 

8 

2 

0 

0 

6. 

equal  in  number  to  the  number  of  system  functions. 

1 

1 

3 

5 

6 

7. 

seldom,  if  ever,  adequately  prepared. 

6 

8 

2 

0 

0 

8. 

easy  to  write  once  you  know  precisely  what  is 
required. 

2 

6 

3 

4 

1 

9, 

easy  to  write  for  a system  currently  in  fleet  use. 

2 

6 

2 

4 

2 

- CONTINUE  WITH  TABLE  B1  - 
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Table  B1  (Continued) 

Functional  requirements  are: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

10.  defined  by  the  operations  one  conducts  to  produce 
them. 

2 

1 

6 

5 

2 

11.  definable  only  by  examples. 

0 

0 

5 

6 

5 

12.  statements  of  goals,  objectives  or  purposes  of  a 
system. 

4 

8 

2 

1 

1 

13.  difficult  to  define  so  that  anyone  with  a typical 
high  school  education  and  access  to  a dictionary 
can  understand  the  definition. 

1 

5 

3 

4 

3 

14.  mandatory  in  system  development  since  they  and  they 
alone  are  fair  standards  by  which  the  adequacy  of 
the  system  can  be  evaluated. 

11 

4 

1 

0 

0 

15.  mandatory  for  development  of  any  system  when  at  least 
one  human  is  to  be  a component  of  the  system. 

11 

4 

1 

0 

0 

16.  of  no  value  after  a system  has  been  delivered  to  the 
fleet . 

0 

2 

2 

3 

9 

17.  neither  good  nor  bad;  rather,  functional  requirements 
are  either  useful  or  useless. 

1 

3 

4 

6 

2 

1 

18.  useful  if  they  facilitate  the  work  for  which  they 
are  needed. 

12 

4 

0 

0 

0 

- CONTINUE  WITH  TABLE  B1  - 
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Table  B1  (Continued) 

Functional  requirements  are: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

19.  useless  if  they  do  not  facilitate  or  they  hinder 
the  work  for  which  they  are  needed. 

11 

3 

0 

2 

0 

20.  different  for  different  uses. 

2 

7 

3 

2 

2 

21.  always  a means  to  an  end. 

5 

9 

2 

0 

0 

22.  always  the  same  for  a given  system. 

3 

1 

0 

7 

5 

23.  never  written  once  in  the  development  of  a system; 

rather  they  are  written  at  the  start  and  continuously 
refined  as  more  information  becomes  available. 

3 

11 

0 

2 

0 

24.  "activities  a system  must  perform  independent  of  the 
means  of  accomplishing  the  activities." 

10 

6 

0 

0 

0 

25.  "mandatory  attributes  that  a system  must  possess  to 
accomplish  its  defined  mission." 

9 

4 

2 

1 

0 

26.  "what  a thing  (system,  display,  control)  is  to 
accomplish. " 

10 

4 

2 

0 

0 

27.  mandatory  for  at  least  one  of  the  human  factors 
support  services  in  system  development. 

11 

1 

4 

0 

0 

- CONTINUE  WITH  TABLE  B1  - 
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Table  B1  (Continued) 

Functional  requirements  are: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

28.  statements  of  WHAT  a system  must  accomplish. 

11 

5 

0 

0 

0 

29.  statements  of  WHEN  a system  must  accomplish 
something . 

4 

4 

1 

4 

3 

50.  statements  of  WHO  should  accomplish  system 
functions . 

0 

1 

0 

j 

10 

5 

31.  statements  of  WHERE  (the  environment)  the 
functions  must  be  accomplished. 

6 

5 

2 

5 

0 

32.  statements  of  WHY  the  functions  must  be 
accomplished. 

3 

2 

5 

2 

4 

33.  written  to  a level  of  detail  determined  by  the  use 
to  which  the  functional  requirements  will  be  put. 

8 

8 

0 

0 

0 

34.  not  complete  unless  all  constraints  that  influence 
each  function  are  described. 

1 

10 

2 

2 

1 

35.  useful  only  in  some  subsequent  human  factors 
support  service. 

0 

0 

1 

6 

9 

36.  not  complete  unless  all  potential  inputs  to  and  ex- 
pected output  of  each  function  are  described. 

1 

9 

4 

1 

1 

- CONTINUE  WITH  TABLE  B1  - 
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Table  B1  (Continued) 

Functional  requirements  are: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

37.  like  the  responses  in  a parlor  game  where  participants 
try  to  describe,  for  example,  all  the  things  a Xerox 
machine  can  do  without  giving  away  that  they  have  a 

Xerox  machine  in  mind. 

2 

5 

6 

1 

2 

38.  related  to  the  concept  of  ^degrees  of  freedom"  in 
that  as  equipment  is  ever  more  precisely  defined, 
the  corresponding  loss  in  alternative  approaches 
to  accomplish  system  functions  is  similar  to  a 
reduction  in  "degrees  of  freedom." 

3 

11 

0 

1 

1 

- END  OF  TABLE  B1  - 
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Table  B2 

Functional  requirements  should  be  written  by: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

1.  non-HF  engineers  for  use  by  HF  engineers. 

1 

j 

h 

i 

i 

4 

1 

6 

2 

2.  HF  engineers  for  use  by  HF  engineers. 

0 

■ 0 1 
! 

1 i 

1 4 i 
I 

I 

i 

1 

1 S 

1 

1 

3 

5.  HF  engineers  for  use  by  non-HF  engineers. 

0 

IJ 

1 

1 

1 

1 

! 4 

1 

1 

7 

3 

4.  someone  with  a background  in  both  engineering 
and  psychology. 

2 

} ' 

1 j 

1 

1 8 

1 1 

1 

1 

3 

1 

2 

0 

5.  someone  with  at  least  one  year’s  experience 
in  the  systems  development  process. 

3 

9 

1 

0 

1 

2 

1 

6.  someone  who  understands  thoroughly  the  warfare  area 
(e.g.,  ASW,  air-to-air  combat)  for  which  the  system 
is  being  developed. 

2 

12 

1 

0 

0 

7.  anyone  who  understands  what  is  needed  and  has  the  skill, 
interest  and  ability  to  think  and  write  clearly. 

5 

9 

2 

0 

2 

8*  anyone  who  has  written  adequate  functional  require- 
ments at  least  once. 

1 

7 

2 

6 

0 

9.  anyone  working  under  supervision  of  an  experienced 
functional  requirements  v\/riter. 

1 

6 

3 

4 

2 

- CONTINUE  WITH  TABLE  B2  - 
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Table  B2  (Continued) 

Functional  requirements  should  be  written  by: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

10.  almost  anyone  who  wants  to  write  functional  require- 

ments since  the  process  is  relatively  simple. once  you 
know  what  is  required  and  considered  adequate. 

0 

1 

1 

8 

i 

6 

11.  individuals  who  will  use  the  functional  requirements. 

0 

2 

10 

4 

0 

12.  system  development  management  personnel. 

0 

5 

4 

5 

2 

15.  active  duty  fleet  personnel  who  represent  people 
who  will  ultimately  use  the  system. 

0 

2 

3 

9 

2 

14.  a group  of  writers  that  represent  all  the  different 
interests  in  the  system  development  process. 

2 

14 

0 

0 

0 

- END  OF  TABLE  B2  - 
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Table  B3 

Functional  requirements  are  the  same  or  nearly 
the  same  as : 

1 

! 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

1. 

Operational  Requirements . 

0 

! 

i 

1 1 

i 

1 

- — f 

5 

4 

i 

5 ! 

2. 

Specific  Operational  Requirements. 

0 

I 

^ i 

1 

1 

4 

4 

4 

5 . 

Development  Concept  Papers. 

0 

0 

1 

1 

11 

4 

4. 

Proposed  Technical  Approaches. 

0 

1 

2 

7 

5 

5. 

Mission  Element  Need  Statements. 

0 

5 

7 

2 

1 

6. 

Specific  Behavioral  Objectives. 

0 

6 

1 

6 

3 

7. 

entries  in  a task  analysis. 

0 

2 

4 

5 

4 

8. 

entries  in  a functional  flow  block  diagram. 

1 

6 

2 

4 

2 

9. 

entries  in  an  operational  sequence  diagram. 

1 

3 

3 

6 

2 

- CONTINUE  WITH  TABLE  B3  - 
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Table  B3  (Continued) 

Functional  requirements  are  the  same  or  nearly 
the  same  as : 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

10.  outcomes  of  a function  allocation  procedure. 

0 

4 

1 

5 

5 

11.  outcomes  of  a methods  and  media  analysis. 

0 

2 

4 

3 

6 

12.  operator/maintainer  roles. 

0 

1 

3 

8 

3 

13.  operator/maintainer  duties. 

0 

1 

3 

8 

3 

14.  operator/maintainer  tasks. 

0 

4 

1 

5 

5 

15.  statements  of  Navy  needs. 

0 

2 

3 

9 

2 

- END  OF  TABLE  B3  - 
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Table  B4 

1 

! 

1 

1 

Likely  sources  of  information  needed  to  write 
functional  requirements  include: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

1. 

Operational  Requirements . 

9 

[ 

7 

0 

0 

0 

2 . 

Specific  Operational  Requirements. 

8 

8 

0 

0 

0 

3. 

Development  Concept  Papers, 

3 

1 

9 

3 

0 

0 

4. 

Proposed  Technical  Approaches. 

0 

9 

6 

0 

0 

5. 

Mission  Element  Need  Statements. 

3 

8 

4 

0 

0 

6. 

Specific  Behavioral  Objectives. 

2 

5 

4 

5 

0 

7. 

entries  in  a task  analysis. 

0 

5 

2 

6 

3 

8. 

entries  in  a functional  flow  block  diagram. 

0 

7 

3 

2 

3 

9. 

entries  in  an  operational  sequence  diagram. 

0 

7 

2 

3 

3 

- CONTINUE  WITH  TABLE  B4  - 
- Bll  - 
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Table  B4  (Continued) 

Likely  sources  of  information  needed  to  write 
functional  requirements  include: 

I AGREE  COME^LETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

10.  outcomes  of  a function  allocation  procedure. 

1 

i 

0 

1 

1 4 

2 

3 

6 

11.  outcomes  of  a methods  and  media  analysis. 

0 

1 

7 

1 

6 

12.  operator/ maintainer  roles. 

1 

4 

4 

4 

3 

13.  operator/ maintainer  duties. 

2 

3 

3 

4 

4 

14.  operator/maintainer  tasks. 

2 

3 

2 

4 

5 

15.  statements  of  Navy  needs. 

4 

9 

2 

1 

0 

16.  interviews  with  non-human  factors  engineers  directly 
involved  in  the  system  development  process. 

3 

12 

1 

0 

0 

17.  experience  gained  by  direct,  daily  involvement  of  the 
writer (s)  with  non-HF  engineers  on  the  development 
team. 

2 

11 

1 

1 

1 

18.  almost  any  source  of  information  since  it  is  impossible 
to  anticipate  where  the  proper  information  can  be 
obtained. 

4 

6 

0 

2 

4 

- END  OF  TABLE  B4  - 
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Table  B5 

Human  factors  support  functions  that  require  adequate 
functional  requirements  include: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

1. 

crew  station  design. 

1 

11  j 

3 

0 

1 

0 

panel  design. 

1 

1 

1 

1 

i 

12 

I 

1 ] 

1 

1 

2 j 

1 

1 ■ ■ ■ \ 

0 

1 

1 

1 i 

1 0 

3. 

control  design. 

1 

1 

1 1 

1 

i 12 

! 

1 

2 

I 

1 

i 0 

1 

0 

4. 

display  design. 

1 

i 12 

1 

3 

1 

1 » 

j 

0 

0 

5. 

function' allocation  procedures. 

13 

2 

1 

1 

0 

0 

0 

6. 

operator  role  definition. 

12 

1 

t 

2 

1 

0 

0 

7. 

maintainer  role  definition. 

10 

3 

1 

1 

0 

8, 

habitability  and  safety  analysis. 

4 

7 

2 

2 

0 

9. 

functional  flow  block  diagramming. 

9 

6 

0 

0 

0 

- CONTINUE  WITH  TABLE  B5  - 
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Table  B5  (Continued) 

Human  Factors  support  functions  that  require  adequate 
functional  requirements  include: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

10.  operational  sequence  diagramming. 

7 

i 

8 

0 

0 

0 

11.  anthropometric  analyses. 

3 

1 

6 

5 

0 

12.  selection  of  equipment  components. 

7 

8 

0 

' 0 

0 

13.  construction  of  time-line  analyses. 

5 

10 

0 

0 

0 

14.  crew  size  determinations. 

10 

5 

0 

0 

0 

15.  task  analyses. 

9 

6 

0 

0 

0 

16.  operational  trainer  design. 

8 

4 

2 

1 

0 

17.  maintenance  trainer  design. 

7 

5 

2 

1 

0 

18.  accident  analyses. 

3 

1 

8 

3 

0 

- CONTINUE  WITH  TABLE  B5  - 
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Table  B5  (Continued) 


Human  factors  support  functions  that  require  adequate 
functional  requirements  include: 


19.  system  test  and  evaluation. 
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- END  OF  TABLE  B5  - 
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Table  B6 

Functional  requirements  should  be  written  without 
reference  to: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

1.  equipment.  Thus,  "Destroy  enemy  forces"  is  a better 

FR  than  "Destroy  enemy  submarines . " 

1 

0 

1 

0 

1 

7 

8 

2.  equipment  specified  to  implement  an  intended  function. 
Thus,  "Destroy  enemy  submarines"  is  a better  FR  than 
"Destroy  enemy  submarines  with  torpedoes." 

-T  i 

5 

2 

3 

3 

3.  specific,  currently  available  equipment.  Thus,  "Destroy 
enemy  submarines  with  torpedoes"  is  a better  FR  than 
"Destroy  enemy  submarines  with  the  MK  46  torpedo." 

3 

5 

2 

4 

2 

4.  outcomes  of  an  implied  function  allocation  process. 

Thus,  "Accomplish  MK  46  torpedo  launch"  is  a better  FR 
than  "Pilot  accomplish  MK  46  torpedo  launch." 

5 

9 

2 

0 

0 

5.  specific  human  responses.  Thus,  "Pilot  launch  MK  46 

torpedo"  is  a better  FR  than  "Pilot  press  LAUNCH  button 
on  a torpedo  control  panel." 

5 

9 

1 

1 

0 

6.  specific  human  responses  with  specific  interface  equip- 
ment. Thus,  "Pilot  press  LAUNCH  button  on  a torpedo 
control  panel"  is  a better  FR  than  "Pilot  press  LAUNCH 
button  on  MK  46  Torpedo  Control  Panel." 

5 

7 

3 

0 

1 

- END  OF  TABLE  B6  - 
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Table  B7 


PLEASE  NOTE;  In  items  1 through  7 inclusive,  your 
opinion  is  expressed  in  the  appropriate  box  as  before 
for  the  part  of  the  item  preceding  the  two  dashes. 

For  example,  in  item  1.  you  check  your  opinion  about 
the  "size”  (usually  measured  in  terms  of  dollar  cost) 
of  the  system--...."  You  can,  if  you  want  to,  pro- 
vide some  bonus  information  by  circling  or  under- 
lining one  of  two  words  within  brackets.  The  word 
you  select  as  appropriate  in  the  part  of  the  item 
after  the  dashes  should  not,  of  course,  influence 
your  opinion  about  the  item. 


The  probability  that  adequate  functional  requiremements 
can  be  written  depends,  in  part,  on  the; 
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1— 1 

1— 1 

l-H  ■ 

1.  size  (usually  measured  in  terms  of  dollar  cost)  of 
the  system— the  (larger;  smaller)  the  system,  the 
higher  the  probability  adequate  FRs  can  be  written. 

! 

2 

! 

1 

7 

1 ■ 

i 

! 

I ^ 

1 

1 

3 

! 

1 

2.  number  of  people  intended  to  operate/maintain  the 
system--the  (more;  fewer)  people  needed,  the  higher 
the  probability  adequate  FRs  can  be  written. 

2 

1 

5 

1 

4 

4 

1 

3.  point  in  the  development  process  when  FRs  are  written- - 
the  (earlier;  later)  in  the  process,  the  higher 
the  probability  adequate  FRs  can  be  written. 

1 

1 

7 

1 

i 

8 

1 

0 

0 

4.  number  of  people  involved  in  writing  FRs--the  (more; 
fewer)  people  involved,  the  higher  the  probability 
adequate  FRs  will  be  written. 

2 

6 

4 

3 

0 

5.  number  of  people  assigned  to  the  system  development 
team--the  (more;  fewer)  people  assigned,  the  higher 
the  probability  adequate  FRs  will  be  written. 

1 

6 

6 

2 

0 

6.  complexity  (use  your  own  interpretation  of  complexity) 
of  the  system — the  (more;  less)  complexity,  the  higher 
the  probability  adequate  FRs  will  be  written. 

3 

8 

4 

1 

0 

- CONTINUE  WITH  TABLE  B7  - 
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Table  B7  (Continued) 


The  probability  that  adequate  functional  requirements 
can  be  written  depends,  in  part,  on  the: 
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7.  criticality  (use  your  own  interpretation  of  criticality) 
of  the  human  role  in  the  system — the  (more;  less) 
critical  the  role,  the  higher  the  probability  adequate 
FRs  will  be  written. 


- END  OF  TABLE  B7  - 


Note — Results  of  responses  to  clauses  after  the  two  dashes  were: 
Question# 


1 

no 

response=6; 

smaller=9;  larger=l 

2 

no 

response=8; 

fewer=8 

3 

no 

response=l ; 

earlier=13;  later=2 

4 

no 

response=8; 

fewer=5;  more=3 

5 

no 

response=8 ; 

f ewer=4 ; more=4 

6 

no 

response=5; 

less=10;  more=l 

7 

no 

response=7 ; 

less=4;  more=5 

Due  to  the  large  number  of  no  response  cases,  results  were  disregarded. 
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Table  B8 


A document  of  instructions  for  writing  functional 
requirements : 


>H 
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hJ 

cu 

o 

u 

tu 

UJ 

a 

< 


1.  could  be  written. 


does  not  exist  presently. 


if  well  written  and  understandable  to  individuals 
involved  in  system  development,  would  be  useful. 


4.  is  not  worth  the  effort  required  to  produce  it. 


is  not  needed  since  the  procedure  for  writing 
functional  requirements  is  an  art  similar  to  writing 
musical  masterpieces  and  thus  cannot  be  described. 


is  not  needed  since  the  procedure  for  writing 
functional  requirements  is  well  known. 


is  a waste  of  time  since  even  a good  document  would  not 
be  used  by  individuals  on  the  system  development  team. 
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8.  is  impossible  to  write  since  it  is  unlikely  two  people 
of  equivalent  backgrounds,  using  the  document,  would 
produce  essentially  identical  functional  requirements. 


9.  is  impossible  to  write  since  the  procedure  varies  too 
much  from  one  system  to  another. 


- CONTINUE  WITH  TABLE  B8  - 
- B19  - 
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Table  B8  (Continued) 

A document  of  instructions  for  writing  functional 
requirements : 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

10.  would  be  a set  of  step-by-step  procedures. 

1 

8 

4 

3 

0 

11.  would  describe  a very  complex  process  and  require 
special  skill,  experience  and  training  to  follow. 

0 

6 

1 

7 

2 

12.  would  be  little  different  from  a text  devoted  to 
discussions  of  thinking  and  writing  clearly. 

0 

3 

2 

'■  9 

2 

13.  would  not  provide  instructions  for  a procedure  or 

process  to  follow;  rather,  the  document  would  provide 
general  guidelines  for  writing  functional  requirements. 

1 

4 

2 

8 

1 

- END  OF  TABLE  B8  - 
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Table  B9 

A document  of  instructions  for  writing  functional 
requirements,  if  one  is  written,  should  include  a 
section: 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

1.  clearly  defining  functional  requirements  and  the 
general  issues  involved  in  writing  good  ones. 

12 

4 

0 

0 

0 

2.  on  the  need  for  clarity  in  thinking  and  writing  along 
with  suggestions  and  references  for  achieving  clarity. 

4 

9 

3 

0 

0 

3.  of  a generous  number  of  examples  of  adequate,  marginal 
and  inadequate  FRs  along  with  explanations  why  each  of 
the  examples  was  classified  as  it  was. 

10 

6 

0 

0 

0 

4.  describing  the  various  uses  HF  engineers  have  for 
functional  requirements . 

4 

10 

1 

1 

0 

5.  describing  the  kinds  and  sources  of  information 

required  to  write  adequate  functional  requirements. 

10 

6 

0 

0 

0 

6.  describing  the  kinds  of  problems  typically  encountered 
in  writing  functional  requirements  along  with  sugges- 
tions for  overcoming  the  problems. 

9 

6 

1 

0 

0 

7.  providing  an  example  of  how  an  existing  system  evolved 
and  where  in  the  evolution  functional  requirements 
would  have  been  needed,  what  the  functional  require- 
ments should  have  looked  like  and  how  they  would  have 
been  used  by  human  factors  engineers. 

8 

7 

1 

0 

0 

- END  OF  TABLE  B9  - 
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Table  BIO 

Version  presented  to  HFEs 

Most  non-human  factors  engineers  believe 
human  factors  engineers : 

Version  presented  to  nHFEs 

Human  Factors  Engineers  (or  Engineering 

Psychologists) : 

. I AGREE  COMPLETELY. 

, I TEND  TO  AGREE. 

j 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

1.  should  have  formal  training  as  engineers  and  HFEs 

then  get  some  additional  training  in  psychology. 

nHFEs 

1 

1 

8 

14 

2 

2 

4 

13 

1 

3 

2.  don't  provide  anything  more  than  common-sense 

4 

7 

2 

2 

1 

suggestions  to  the  system  development  process.  ' 

i 

0 

3 

1 

19 

9 

3.  provide  services  that  cost  too  much  money 

5 

8 

0 

3 

0 

to  be  worthwhile  in  the  development  process. 

0 

1 

2 

18 

12 

4.  provide  services  that  are  too  time-consuming 

4 

7 

3 

2 

0 

to  be  worthwhile  in  the  development  process. 

0 

2 

4 

15 

12 

5.  can't  respond  to  system  development  problems 

2 

12 

0 

2 

0 

without  first  proposing  a lengthy  "investi- 

gation." 

2 

12 

4 

12 

3 

6.  have  little  or  nothing  of  value  to  contribute 

0 

6 

5 

3 

2 

to  the  system  development  process. 

0 

1 

0 

10 

22 

7.  promise  a great  deal  but  deliver  very  little. 

2 

10 

2 

1 

1 

0 

7 

6 

10 

10 

8.  use  a lot  of  jargon  that  even  other  human 

0 

6 

6 

2 

2 

factors  engineers  do  not  really  understand. 

0 

3 

4 

11 

13 

- CONTINUE  WITH  TABLE  BIO  - 
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Table  BIO  (Continued) 

Version  presented  to  HFEs 

Most  non-human  factors  engineers  believe 
human  factors  engineers : 

Version  presented  to  nHFEs 

Human  Factors  Engineers  (or  Engineering 

Psychologists) : 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

9.  have  a useful  role  in  the  system  development  HFEs 

0 

6 

2 

7 

1 

process . 

nHFEs 

18 

14 

0 

0 

1 

10.  want  a role  in  system  development,  but 

1 

9 

1 

4 

1 

when  they  are  given  a role  they  don*t  know 

what  to  do  next . 

1 

5 

8 

12 

7 

11.  are  insensitive  to  demands  and  responsibilities 

0 

11 

2 

3 

0 

non-human  factors  engineers  face  in  system 

development . 

1 

9 

6 

9 

6 

12.  are  unaware  of  the  factors  that  really  determine 

3 

5 

3 

5 

0 

events  and  decisions  in  the  system  development 

process . 

1 

9 

3 

13 

7 

15.  are  arrogant. 

0 

1 

6 

6 

3 

0 

2 

2 

13 

14 

14.  are  not  good  "team  players." 

0 

6 

3 

6 

1 

0 

4 

4 

12 

12 

15.  work  in  system  development  only  against  their 

0 

1 

3 

10 

2 

will . 

0 

1 

9 

15 

7 

16.  work  in  system  development  because;  a)  the  pay  is 

0 

1 

6 

7 

2 

good,  b)  the  work  is  easy  and  c)  accountability  for 

their  performance  does  not  exist. 

0 

0 

8 

14 

9 

- CONTINUE  WITH  TABLE  BIO  - 
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Table  BIO  (Continued) 

Version  presented  to  HFEs  i 

Most  non-human  factors  engineers  believe  j 

human  factors  engineers : 

Version  presented  to  nHFEs 

Human  Factors  Engineers  (or  Engineering 

Psychologists) : i 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

17.  would  rather  do  research  in  human  behavior  HFEs 

0 

2 

9 

4 

1 

than  work  in  system  development. 

nHFEs 

2 

8 

12 

7 

1 ' 

18.  are  useful  only  to  suggest  arrangements  for 

3 

8 

3 

1 

i 

! 1 

the  location  of  controls  and  displays  at  the 

man-machine  interface . 

1 

3 

2 

21 

6 

1 

19.  are  welcomed  in  the  system  development  process. 

0 

2 

2 

11 

1 

2 

16 

5 

7 

2 

20.  try  to  "horn  in"  on  a process  where  they  are 

0 

11 

1 

4 

0 

not  wanted. 

0 

1 

3 

22 

5 

21.  are  unwilling  to  "work  in  the  trenches" 

1 

7 

4 

3 

1 

with  working  level  non-human  factors  engineers. 

0 

5 

4 

13 

9 

22.  should  not  have  responsibility  for  deciding  if 

0 

10 

3 

2 

1 

a human  or  a machine  should  accomplish  a given 

system  function. 

2 

7 

2 

14 

6 

23.  overemphasize  the  role  of  the  human  in  the 

1 

10 

1 

4 

0 

system. 

1 

3 

1 

21 

6 

24.  should  be  involved  in  the  system  development 

3 

10 

0 

2 

1 

process  but  only  after  major  equipment 

components  have  been  selected. 

1 

1 

0 

19 

12 

- CONTINUE  WITH  TABLE  BIO  - 
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Table  BIO  (Continued) 
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Version  presented  to  HFEs 

Most  non-human  factors  engineers  believe 

human  factors  engineers : 


Version  presented  to  nHFEs 

Human  Factors  Engineers  (or  Engineering 

Psychologists) : 


25.  are  not  satisfied  with  the  non-human 
factors  engineers*  selection  of  inter- 
face components. 
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HFEs 

nHFEs 


0 11  4 
2 11  11 


1 0 
6 1 


26.  are-  least  welcome  at  the  concept  formu- 

1 

7 

4 

3 

1 

lation  state  of  the  development  process. 

4 

8 

5 

12 

4 

27.  may  be  O.K.,  but  the  ones  I have  actually 

0 

10 

3 

3 

0 

worked  with  did  not  live  up  to  expectations. 

0 

6 

4 

9 

12 

28.  would  rather  work  with  other  human  factors 

0 

— 

6 

6 

3 

1 

engineers  than  with  non-human  factors 
engineers . 

1 

9 

11 

6 

1 

29.  are  useful  only  in  support  services,  (e.g.. 

2 

8 

4 

2 

0 

training  system  design,  test  and  evaluation) 

after  prototype  models  of  the  system  have  been  built. 

0 

0 

1 

16 

16 

30.  should  be  restricted  to  designing  comfortable 

2 

5 

7 

2 

0 

and  attractive  crew  stations. 

0 

0 

2 

13 

18 

51.  do  not  appreciate  the  speed  with  which  things 

3 

11 

1 

1 

0 

happen  during  the  system  development  process. 

3 

11 

5 

9 

5 

32.  should  not  have  final  authority  in  any  decision 

5 

9 

2 

0 

0 

in  the  development  process. 

4 

9 

4 

11 

3 
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Table  BIO  (Continued) 


Version  presented  to  HFEs 

Most  non-human  factors  engineers  believe 

human  factors  engineers: 


Version  presented  to  nHFEs 

Human  Factors  Engineers  (or  Engineering 

Psychologists) : 


53.  treat  each  system  problem  they  are  asked 
to  help  resolve  as  if  it  were  the  first 
time  the  problem  had  ever  come  up. 

34.  are  responsible  for  any  inadequacies  in 
communication  or  cooperation  between  non- 
human factors  and  human  factors  engineers . 


- END  OF  TABLE  BIO 


UJ 

W 

a: 

< 

CO 

HH 

Q 

OC 

O 

1 

cu 

i 

a: 

• 

>- 

< 

• 

• 

UD 

>H 

i— H 

CD 

H 

CD 

tu 

UJ 

. 1 

CL. 

ci 

U 1 

1— i 

o. 

tu 

U 1 

< 

s 

Z 

C/D 

o 

Cu 

1— ( 

HH 

u 

< 

< 

Q 

O 

CU’ 

u 

o 

o 

UJ 

cu 

ci 

tu 

u 

cu 

Q 

z 

Q 

< 

2: 

'Z 

CO 

tu 

UJ 

1— i 

< 

S 

Q 

1— i 

HH 

1 

HH 

HH 

H- ( 

HFEs 

1 
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0 

nHFEs 

1 

9 

3 

18 
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0 

3 

10 

2 

1 

0 

0 

5 

13 

14 
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Table  Bll 

Most  human  factors  engineers  believe  non-human 
factors  engineers : 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

T DISAGREE  COMPLETELY. 

1.  are  unable  to  think  in  abstract  terms. 

2 

5 

4 

4 

1 

2.  are  incapable  of  thinking  in  new  and  novel 
terms. 

1 

8 

1 

5 

1 

3.  talk  in  jargon  even  other  non-HF  engineers 
don't  understand. 

0 

6 

6 

4 

i 

i 

0 

4.  have  probably  chosen  to  become  engineers  because 
they  are  uncomfortable  with  the  unpredictability 
they  believe  is  the  characteristic  of  human  behavior. 

1 

4 

6 

3 

2 

5.  are  arrogant. 

0 

4 

3 

8 

1 

6.  are  unconvinced  about  the  usefulness  of  an  HF 
discipline . 

2 

14 

0 

0 

0 

7.  are  convinced  the  HF  discipline  has  some,  but  vague, 
role  in  the  system  development  process. 

0 

13 

2 

1 

0 

8.  are  willing  to  give  only  "lip  service"  to  the  need 
for  HF  involvement  in  the  system  development  process. 

2 

14 

0 

0 

0 

9.  are  afraid  HF  engineers  will  interfere  with  the  non-HF 
engineer's  prerogative  to  make  system  design  decisions. 

1 

12 

3 

0 

0 
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Table  Bll  (Continued) 

Most  human  factors  engineers  believe  non-human 
factors  engineers : 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

10.  do  not  appreciate  the  complexity,  time  and  effort 
involved  in  human  factors  support  services. 

6 

9 

1 

0 

0 

11.  are  unaware  of  the  reasons  for  the  lack  of 
generality  in  the  human  factors  data  base. 

4 

8 

4 

0 

0 

12.  when  all  is  said  and  done,  must  have  the 

principal  authority  to  decide  on  the  system 
that  will  be  delivered  to  the  fleet. 

0 

11 

2 

2 

1 

13.  are  biased  to  think  in  terms  of  equipment 
solutions  to  system  functions. 

8 

7 

1 

0 

0 

14.  want  to  develop  new  systems  by  improving  the 
old  ones. 

0 

13  • 

2 

1 

0 

15.  do  not  know  how  and  why  HF  engineers  do  what 
they  do. 

2 

12 

2 

0 

0 

16.  would  not  change  their  beliefs  even  if  they 
, were  shown  their  beliefs  are  based  on  mis- 
interpretation of  fact. 

0 

5 

3 

7 

1 

17.  cannot  think  of  ways  to  accomplish  system 

functions  unless  they  start  their  thinking  with 
some  piece  or  type  of  equipment  in  mind. 

3 

9 

1 

3 

0 

18.  are  less  intelligent  than  HF  engineers. 

2 

1 

4 

6 

3 
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Table  Bll  (Continued) 

Most  human  factors  engineers  believe  non-human 
factors  engineers : 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

19.  have  had  less  training  and  practice  at  verbalizing 
their  ideas  than  human  factors  engineers. 

1 

9 

3 

3 

0 

20.  should  be  reduced  to  a very  low  level  in  the 
decision-making  process  in  system  development. 

0 

0 

3 

I 

10 

3 

21.  are  quick  to  cut  funds  for  human  factors  support 
services  from  the  budget  when  funding  is  cut. 

6 

9 

1 

0 

0 

22.  assign  as  many  functions  as  possible  to  equipment 
and  assign  to  humans  only  what  unavoidably  remains. 

4 

9 

1 

2 

0 

23.  assume  humans  will  adapt  to  the  demands  of  the  man- 
machine  interface  since  humans  have  always  done  so 
in  the  past. 

4 

12 

0 

0 

0 

24.  assume  training  will  take  care  of  helping  humans 
adapt  to  any  demands  the  system  might  impose. 

4 

12 

0 

0 

0 

25.  look  forward  to  the  day  when  all  system  functions 
can  be  accomplished  by  machines  and  human  factors 
problems  will  not  exist. 

0 

7 

6 

2 

1 

26.  rely  on  the  difficulty  of  judging  adequacy  of 
human  performance  to  escape  accountability  for 
equipment  design  decisions. 

1 

8 

5 

2 

0 

27.  should,  if  they  are  going  to  interact  with  HF 
engineers,  have  at  least  some  basic  training 
in  psychology. 

0 

4 

5 

7 

0 
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Table  Bll  (Continued) 


Most  human  factors  engineers  believe  non-human 
factors  engineers : 
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28.  ^’hold  all  the  trump  cards’’  since  non-human  factors 
engineers  make  all  final  decisions  in  the  develop- 
ment process. 

1 

! 2 

10 

4 

0 

0 

i 

29.  are  not  good  "team  players." 

0 

1 

5 

10 

1 

0 

30.  do  not  feel  any  responsibility  for  the  adequacy  of 
human  performance  in  the  developed  system  when  it 
is  delivered  to  fleet. 

1 

14 

0 

1 

1 

0 

1 

1 

i 

51.  are  not  satisfied  with  human  factors  engineering 
solutions  for  system  design  problems. 

I 

I 

1 

9 

4 

2 

0 

32.  will  abandon  even  an  excellent  HF  engineering 
suggestion  if  the  suggestion  interferes  with  a 
constraint  imposed  by  the  needs  of  the  equipment. 

3 

13 

0 

0 

0 

33.  have  no  intentions  of  letting  human  factors  engineers 
determine  the  form  of  the  man-machine  interface. 

0 

8 

2 

6 

0 

34.  would  prefer  to  work  with  other  non-human  factors 
engineers  rather  than  with  human  factors  engineers. 

2 

10 

3 

1 

0 

55.  are  responsible  for  any  inadequacies  in  communication 
or  cooperation  between  non-HF  and  HF  engineers. 
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6 
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Table  Bll  (Continued) 

Most  human  factors  engineers  believe  non-human 
factors  engineers : 

I AGREE  COMPLETELY. 

I TEND  TO  AGREE. 

I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE. 

I TEND  TO  DISAGREE. 

I DISAGREE  COMPLETELY. 

36.  are  jealous  of  their  "turf”  and  don^t  want 
to  lose  any  control. 

1 

11 

2 

2 

0 

37.  are  unaware  that  they  are  not  doing  what  the  human 
factors  engineers  think  they  should  be  doing  when 
non-HF  and  HF  engineers  cooperate  in  system  develop- 
ment . 

0 

7 

7 

2 

0 

38.  when  they  become  managers  of  system  development, 
remain  biased  against  human  factors  engineering. 

1 

10 

0 

5 

0 

39.  are  convinced  HF  engineering  is  too  young  a discipline 
to  be  useful  in  solving  today's  system  development 
problems . 

0 

4 

6 

5 

1 
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Appendix  C 
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Sixteen  HFEs  and  33  nHFEs  responded  to  questionnaire  items.  For  each 
item,  respondents  selected  one  from  among  five  categories,  viz.: 

= I AGREE  COMPLETELY 

= I TEND  TO  AGREE 

C.,  = I'M  UNCERTAIN  IF  I AGREE  OR  DISAGREE 
o 

= I TEND  TO  DISAGREE 
= I DISAGREE  COMPLETELY 

The  number  of  responses  in  a distribution  (N)  was  not  always  16  or  33  since 
respondents  could  ignore  any  item  by  not  responding  to  the  item.  Observed  N 
values  in  this  exercise  were  15  or  16  for  HFEs  and  28,  30,  31  or  33  for  nHFEs. 

Response  distributions  for  each  item  represent  opinions  of  the  group  about 
corresponding  items.  The  distribution  Median  represents  one  statistical  summary 
of  group  opinion.  To  compute  distribution  Medians,  class  limits  for  categories 
were  set  at : 


= .500  to  1.499 

= 1.500  to  2.499 

C,  = 2.500  to  3.499 
o 

= 3.500  to  4.499 
= 4.500  to  5.499 

With  the  limits  set  above,  the  following  is  true: 

1.000  = Median  i 5.000 

When  all  cases  are  observed  in  C^,  the  Median  is  1.000.  represents  complete 

agreement  with  the  associated  item.  Thus,  when  the  Median  is  1.00,  all  res- 
pondents indicate  con^lete  agreement  with  the  associated  item.  Likewise,  when 
the  Median  = 5.00,  all  respondents  indicate  complete  disagreement  with  the  item. 
Thus,  distribution  Medians  represent  points  on  an  ordinal  scale  from  complete 
agreement  (Mdn.  = 1.00)  to  complete  disagreement  (Mdn.  = 5.00). 

To  determine  whether  or  not  a distribution  represents  consensus  of  opinion, 
observed  distributions  were  compared  to  distributions  expected  by  chance.  For 
N values  encountered  in  this  exercise  the  probability  of  occurrence,  by  chance, 
of  various  sums  of  frequencies  in  and  or  and  was  determined.  For 

example,  when  16  things  are  distributed  to  five  categories,  4,845  different 
distributions  can  be  formed. ^ For  17  of  the  4,845  distributions  = 16, 

^A  computer  program  for  listing  properties  of  distributions  of  N values  in  this 
exercise  is  available  from  the  author. 
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viz.:  = 16,  = 0;  = 0,  = 16;  = 15,  = 1;  = 1,  = 15;  ... 

Cl  = 9,  C2  = 7;  = 7,  C2  = 9;  = 8,  C2  = 8.  Likewise,  for  17  distributions 

C^+C^  = 16.  Thus,  for  34  of  4,845  distributions  C^+C2  or  = 16.  Con- 

sequently, the  probability  of  randomly  drawing  a distribution  from  among  the 
4,845  possible,  where  C^+C2  or  C^+C^  = 16  is  given  by  34/4,845  or  .007.  In  the 

same  manner,  it  can  be  shown  C^+C^  or  C^+C^  = 15  in  96  of  the  4,845  distributions. 
Thus,  the  probability  of  randomly  drawing  a distribution  where  C^+C2  or 
C^+C^  = 15  is  96/4,845  or  .02.  Consequently,  the  probability  of  randomly  draw- 
ing a distribution  wherein  C^+C2  or  C^+C^  = 15  or  more  is  given  by  .007+. 020  or 

.027.  Table  Cl  shows  probability  values  for  randomly-drawn  distributions  where- 
in Cj^+C2  or  C^+C^  equal  or  exceed  tabled  values  for  N values  encountered  in  this 

exercise. 

Table  Cl  shows  criterion  values,  to  apply  to  observed  distributions,  for 
each  N value.  For  example,  when  N=15,  the  criterion  value  is  13.  Thus,  an 
observed  distribution  where  N=15  and  C^+C2  or  C^+  C^  equals  or  exceeds  13,  is 

taken  to  represent  consensus  of  opinion  for  the  group  with  the  associated  item. 
Criterion  values  for  N=16,  28,  30,  31,  32  and  33  were  13,  24,  26,  26,  28  and 
28  respectively  (See  Table  Cl) . If  the  criterion  value  is  met  or  exceeded  in 
Cj^  and  C2,  consensus  of  opinion  is  for  agreement.  If  the  criterion  value  is  met 

or  exceeded  in  C^  and  C^,  consensus  of  opinion  is  for  disagreement.  Distributions 

wherein  criterion  values  are  not  met  represent  no  consensus  of  opinion  or  un- 
certainty. 
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Table  Cl 


Probability  C^+C2  or  C^+C^  = £ in  a Randomly -Drawn  Distribution 

with  Five  Categories 

(Selected  Values  for  N) 


S 


15 

16 

13 

.074 

.122 

14 

.031 

.064 

15 

.008 

.027 

16 

.007 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 


28  30  31 


32  33 


.104 

.053 

.023 

.008 

.002 


.086 

.044 

.019 

.006 

.001 


.140 

.080 

.041 

.018 

.006 

.001 


.081 

.041 

.018 

.006 

.001 


.118 

.067 

.034 

.015 

.005 

.001 
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